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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133), 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)E3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Ouay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) (E Claim(s) 1-38 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) 03 Claim(s) 1-38 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement 

Application Papers 

9) M The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)DAII b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No, . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 178. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 



Attachment(s) 

1) M Notice of References Cited (PTO-892) 

2) CH Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) S Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 



4) CH Interview Summary (PTO-413) Paper No(s). 

5) □ Notice of informal Patent Application (PTO-152) 

6) □ Other: 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 11-03) 
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DETAILED ACTION 
Specification 

1 . A patent abstract is a concise statement of the technical disclosure of the patent and 
should include that which is new in the art to which the invention pertains. The abstract should 
not refer to purported merits or speculative applications of the invention and should not compare 
the invention with the prior art. The abstract is objected to because the sentences on lines 6-15 
refer to speculative merits of the invention (i.e. "an improved apparatus", "In one advantageous 
embodiment"). 

2. The abstract of the disclosure is objected to because of the following minor informalities: 
the use of the phrase "and capable of on lines 11,12 and 14 is redundant. Correction is 
required. See MPEP § 608.01(b). 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 
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3, Claims 1-7,9, 11-17, 19, 21-26, 28, 30-35 and 37 are rejected under 35 U.S.C. 102(b) as 
being anticipated by the article entitled "Color SuperHistograms for Video Representation", 
written by Dimitrova et al. 

Referring to claims 1, 11,21 and 30, the article teaches an apparatus, system, method and 
computer executable instructions capable of creating a visual summary of video material (page 
316, Figure 1). Specifically, the article teaches the capability of extracting frame signatures 
(histograms) from keyframes and using the frame signatures to create superhistograms, wherein 
the superhistogram represents a visual summary of the video material (scenes), as recited on 
page 314, right column, lines 11-25. 

Referring to claims 2, 12, 22 and 31, the article teaches the filtering of keyframes 
(merging of histograms into family histograms) and extracting frames signatures (computing 
color histograms) from the filtered keyframes before using the frame signatures (histograms) to 
create the superhistogram (page 315, right column, section 2 and page 316, left column, section 
2.3). 

Referring to claims 3, 13, 23 and 32, the article teaches the use of superhistograms to 
cluster the filtered keyframes (the ordered merging of the family histograms to create the 
superhistogram), wherein the clustered keyframes (superhistogram) represents the visual 
summary of the video material, as recited on page 314, right column, lines 1 1-25 and shown in 
Figure 1. 

Referring to claims 4 and 14, the article teaches the use of a histogram as the frame 
signature used to compute superhistograms (page 314, right column, lines 11-15). 
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Referring to claims 5, 15, 24 and 33, the article teaches the use of the LI distance 
measure method, L2 distance measure method, histogram intersection method, Chi-Square test 
and Bin-wise histogram intersection method to computer the histogram difference (page 315, 
right column). 

Referring to claims 6, 16, 25 and 34, the article teaches the selection of the most 
meaningful image for each superhistogram (the top n largest families) as the representative 
image (page 316, section 2.4). 

Referring to claims 7, 17, 26 and 35, the article teaches the ability to select the family 
histograms (the top n largest families) to use to create the superhistogram used to create the 
visual summary (page 316, section 2.4). 

Referring to claims 9, 19, 28 and 37, the article teaches the use of the visual summary 
obtained from the superhistograms to access at least a portion of the video material (classifying 
and searching in video archives), as recited on page 317, section 4.2. 



Claim Rejections - 35 USC§ 103 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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4. Claims 8, 10, 18, 20, 27, 29, 36 and 38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the article entitled "Color SuperHistograms for Video Representation", written 
by Dimitrova et al., in view of Wang et al. U.S. Patent 5,805,733. 

Referring to claims 8, 18, 27 and 36, while the article (written by Dimitrova et al.) 
teaches all of the limitations as applied to the claims above, it fails to teach the capability to 
retrieve a visual summary stored in a memory unit and causing the visual summary to be 
displayed in response to a user request. Wang et al. teach the summarization of video sequences 
similar to that of the article. In addition, Wang et al. further teach the capability of letting a user 
select a visual summary (scene) for viewing, retrieving that visual summary from memory and 
displaying it in response to the user's request, as recited in column 2, lines 16-29 and shown in 
Figure 2. It would have been obvious to one of ordinary skill in the art, having the teachings of 
the article and Wang et al. at the time the invention was made, to modify the video 
summarization scheme disclosed in the article to include the retrieval and display of the visual 
summary in response to a user request, as taught by Wang et al. One would have been motivated 
to make such a combination to give users the flexibility to select which scenes to watch, saving 
them time from having to browse through all of the other irrelevant scenes. 

Referring to claim 10, 20, 29 and 38, while the article (written by Dimitrova et al.) 
teaches all of the limitations as applied to the claims above, it fails to teach the creation of new 
video material using the compact visual summaries. Wang et al. teach the creation of new video 
material (a collage) made up of representative frames for each compact visual summary, as 
recited in column 3, lines 53-57. It would have been obvious to one of ordinary skill in the art, 
having the teachings of the article and Wang et al. at the time the invention was made, to modify 
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the video summarization scheme disclosed in the article to include the creation of new video 
material, as taught by Wang et al. It would have been advantageous for one to utilize such a 
combination in order to conserve processor time and storage space by utilizing the already 
existing visual summaries in the creation of new visual materials. 

5. The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1 . 1 1 1(c) to consider 
these references fully when responding to this action. The documents cited therein teach 
electronic books with similar mechanisms for note taking and retrieval. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ting Zhou whose telephone number is (703) 305-0328. The 
examiner can normally be reached on Monday - Friday 7:00am - 3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-3 1 16. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. / / 



Conclusion 



November 26, 2003 
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